1. Field of the Invention
The present invention generally relates to a method of constructing a neural network. More particularly, the present invention relates to a method of determining the optimal number of neurons contained in hidden layers (also called "intermediate layers") of a feed-forward neural network.
2. Description of the Related Art
In the application of a feed-forward neural network (also called as "multi-layered neural network" and hereinafter referred to as "FFNN") to a specific question, conventionally the number of layers and the number of neurons in the hidden layers have been determined by researchers from their experiences and intuition. As shown in FIG. 9A, the FFNN, which is different from a recurrent type neural network shown in FIG. 9B, forms a one-way transmission network from an input layer to an output layer via at least one hidden layer. In the multi-layered FFNN, Back Propagation (BP) learning has been applied in general. BP learning is a learning algorithm which, when an output value with respect to the input value of a specific corollary or question is obtained, changes link weights (the coefficients of linkage) for linking each layer from the output layer side so that the desired output value with respect to the input value can be outputted for the corollary. According to the conventional method, an FFNN in which the multi-layered composition has provisionally been determined is trained through BP learning, and if the learning error value is smaller than a predetermined value, i.e., if the learning is successful, the FFNN is used for practical application. The employment of this method can be attributed to the fact that not only FFNN's but also general neural networks have never been theoretically analyzed to this date. As neural networks are not always of the optimum structure, sometimes several models of the neural network are prepared and trained and the best network for learning is used.
In any case, even if an, FFNN has been completely trained and is now suitable for practical application, there is no guarantee that this FFNN is optimum for the practical application and has the minimum number of the neurons in the hidden layers. Rather, an FFNN typically has extra neurons in the hidden layers. This means that more calculations than needed are performed, and this requires extra memory size. As a result, there is a problem that the network can handle only a limited range of questions and takes more time for calculation. Another problem is that if the network is formed in hardware, the hardware is dimensionally larger than the hardware needs to be. Accordingly, a method of constructing the optimum neural network has been sought.